Automatic Diacritization for Urdu

Urdu language is written in Arabic script. In this script, the consonantal context is clearly represented, but the vocalic sounds are represented (mostly) by marks or diacritics, which are optional and normally not written. Readers can guess the diacritics and thus can pronounce words correctly, based on their knowledge of the language. But un-diacritized Urdu text creates ambiguity for novice learners and computational systems that require pronunciation. In this paper, a statistical approach is used to mark diacritics for Urdu automatically. Use of multiple knowledge sources is also integrated with the statistical techniques to investigate their effects to the process. These knowledge sources include stemming, part-of-speech tagging, pronunciation lexicons, and word bigrams. The experimental results show that letter-level trigram model performs best and achieves 95.37% overall accuracy by applying all knowledge